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ABSTRACT 


This report describes a project which developed an objective 
measurement of motivation using Flanagan's critical incident 
technique. The sample population was U. S. Navy Jezebel operators. 
The report includes a rating form which was used, the statistical 
computations of the data recorded and a new, modified form. The 
rating form was found to have a high interrater reliability; many 
of the scores on the items had an exceptionally high correlation 
with the total score received and the study generated a particularly 


good operational definition of motivation. 
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I, INTRODUCTION 


The problem of increasing productivity or improving performance 
is one which concerns everyone in a managerial position. Numerous 
studies have been conducted to identify and attempt to control as 
many of the factors that have a bearing on output as possible. One 
of the best descriptions of the interrelated nature of the many 
elements involved can be found in Sutermeister (Sutermeister, 1969). 
In his conceptual scheme the factors most closely related and 
directly influential on productivity are the physical elements of 
the job (i.e., technological development, raw materials, job layout 
and methods) and the employee's job performance which he attributes 
to a complex interaction of ability and motivation. While ability 
1S not a precisely defined or completely understood concept it 1s 
measurable and reliably modifiable. The motivational level of 
employees is a different matter altogether, both in terms of the 
degree of agreement on the elements of motivation and in the 
untapped potential for greater understanding and selective modifi- 
cation of motivation. It can safely be said that there is virtually 
universal acceptance of the important influence that motivation has 
in a broad range of human endeavor. 

In the past 25 years there has been a proliferation of theories 
of motivation. Many of these, such as Maslow's "Need Hierarchy 
Theory,'' (Maslow, 1943),'McGregor's Theory Y'' (McGregor, 1960), 


Victor Vroom's '"Instrumentality Theory" (Vroom, 1964) and Herzberg's 





"Two Factor Theory" (Herzberg, 1959) have generated widespread 
interest. More recently several organizations have come into 
existence that purport to selectively modify the level of motivation 
of individuals, groups, or of entire organizations (for example, 
Success Motivation Institute of Waco, Texas). 

There is one fundamental element that seems to be missing in 
both the theoretical and the applied approaches to motivation. This 
missing element is a method of determining the level of motivation 
(as represented in performance) of a group or an individual. Such 
a technique is necessary to enable investigators to detect differences 
or changes in motivation and to test the effects of incentives or of 
management policy. An intensive search of the available literature 
failed to locate an objective instrument for the measurement of 
motivation. 

It was important in the initial stages of this project to 
limit the area to be investigated and to select a definition of 
motivation upon which to base the study. It was assumed that 
motivation requires force and direction and the study was limited 
to the work context. Motivation then is task related effort expended 
in directed activity. In the investigation of the literature it 
became increasingly obvious that there are two quite distinct 
approaches to motivation, one is the classical psychological view 
of motivation (which refers to the processes governing choices made 
among alternative forms of voluntary activity), the other is 
motivation as seen by the management theorists. This dichotomy seems 
to have developed from the method of approaching the problem. The 


psychologists have looked at physiological states, generally resulting 





from deprivation, and have inferred motivation from observed 
behavior, content analysis, or stated attitudes whereas researchers 
dealing with an industrial or social organization have tended to 
view motivation as the element of performance that can not be 
explained by any other known variable. This investigation was 
oriented along the latter lines, that is the scope was limited to 
the world of work and the motivation to perform a particular task 


: 


or group of tasks. 





II. NATURE OF THE PROBLEM 


In the development of a measurement of motivation, the major 
problem was to determine what behavior is characteristic of a highly 
motivated task-oriented worker. Specifically it was necessary to 
determine whether most motivated men in a reasonably limited job 
category can be expected to be identifiable through a few acts or 
a few classes of characteristic action and whether certain actions 
are characteristic of all or at least a significant percentage of 
highly motivated people doing approximately the same job. The 
second important consideration was to insure that a reasonably 
accurate performance measure was available for the population 
sampled. Failure to consider this element would result in an 
instrument which could not be "calibrated" or tested in a real world 
©; GuingeweAs a larger percentage of jobs have a less direct influence 
on production, this requirement becomes much more difficult to satisfy. 
Because of this, it was necessary to conduct the study outside the 
setting of the production line to insure relevance to modern industry 
and the Armed Forces. 

Whenever a measurement instrument purports to measure a concept 
as 1ll-defined, poorly understood and with the many dimensions of 
motivation, the problems of validity and objectivity become quite 
severe. Is it really motivation that we are measuring or some 
complex and obscure artifact? Do the raters see the behavior of 


the people they are rating as demonstrative of an element of a 
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complex characteristic? How competent are raters at generalization 
from one specific act to a global evaluation? How much is the 
evaluation of motivation influenced by performance? Indeed, how much 
can performance and motivation be separated? 

As a result of these questions it was imperative to select an 
experimental technique to minimize the influence of unwanted variables. 
A unique opportunity and certain necessary encouragement to study this 
problem were presented by Dr. John P. Smith of the United States Naval 
Personnel and Training Research Laboratory, San Diego, California. He 
and his Jezebel Study Group had developed an effective performance 
measure for operator proficiency and had done some work with a 
preliminary motivation rating scheme. As a part of the field testing 
of their proficiency test a set of general demographic information 
was gathered about each man. This included both biographical and 
experience data. As an element of this information, supervisors 
were asked to rate the men working for them on their degree of 
motivation. The instructions for this rating included the comment 
that "motivation implies consistent willingness to give effort to a 
task or duty even in the absence of demand by a military superior; it 
implies a belief in the importance of the Air ASW mission and it 
implies a constructive attitude toward other squadron personnel." 

The men were given a numerical rating from one to seven, this score 
was used for analysis and tests of correlation with the performance 
rating and the other factors. Their analysis revealed that two of 


the factors (rank and motivation) had a significantly higher 








correlation with the performance rating than did the other elements 
and this contribution was not predicted by any of the aptitude or 
other personal characteristics such as GCT, ARI, age or education. 
The following paragraph is an excerpt from their report: 


The guidelines given to the raters to define motivation 
are consistent with characteristics which would be expected to 
be present in career personnel; if so, rank and motivation 
rating should be correlated. Investigation of this point 
resulted in a moderately low Pearsonian correlation coefficient 
of .36. The two variables are related, but they are also 
sufficiently independent of each other to strongly suggest 
that further study of this approach to motivation may be 
profitable. Further data analysis has been carried on to 
evaluate more precisely the significance of the motivation 
ratings uSing analysis of coveriance and multiple correlation. 


(Naval Personnel and Training Research Laboratory Technical Bulletin 


STB 11-5.) 
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III. EXPERIMENTAL PROCEDURE 


A. GENERAL PROCEDURE 

In the previous section a number of the problems inherent in 
developing an instrument for the measurement of motivation were 
discussed. A constant concern in the selection of an appropriate 
procedure and in processing the data that were generated was to 
minimize the influence of these problems. In keeping with this 
philosophy, the method of choice was John C. Flannigan's Critical 
Incident Technique. Very briefly the Critical Incident Technique 
involves describing a situation for the person being interviewed 
and asking him to relate an incident that fits this situation. 
Specifically, in this project the men were asked to, "Think of 
the most highly motivated Jezebel operator that you know. What is 
it about him that caused you to select him? Please be as specific 
as you can."' After asking this question the only additional 
comments that were made by the interviewer were requests for greater 
amplification of the incidents that were reported or for more specific 
incidents (Flannigan, 1954). 

Interviews were conducted on two occasions. The first set of 
interviews were with one senior petty officer from each of four VP 
(Patrol) squadrons and two petty officers from another squadron. 
Since this set represented a horizontal section across the rank 
structure the second interviews involved a vertical section of 


eight operators whose paygrade varied from E3 to E6 (AWAN-AW1) in 


ti 





a Single squadron. It was felt that this procedure would tend to 
minimize the possibility that a significant difference existed 
between the conception of motivated behavior held by junior and 
senior petty officers. The initial intent was to investigate the 
possibility of developing incidents descriptive of highly motivated 
and poorly motivated behavior. This was unsuccessful with the 
interview technique that was used; only positively motivated behavior 
was described by specific incidents. There are several possible 
reasons for this and they are described in the Results and Discussion 
Section. 

A total of 97 incidents were collected and after closely related 
items were combined or eliminated 55 separate items were retained 
for use. Several independent attempts were made to order these 
items into dimensions of motivation and to produce a scale of 
increasingly favorable or graded desirability. These attempts 
were unsuccessful; the incidents did not naturally fall into 
distinct categories and they did not have varying degrees of 
desirability. Twenty items were selected for inclusion on the 
rating form. The basis for selection was the frequency that a 
behavior was reported or the particularly characteristic nature 
of the behavior. It is important to note that in several interviews 
one incident was sufficient to cause a man to be singled out and 
described. In general the behavior that was described in this 
manner represented a considerable amount of time and effort on a 


project or a breakthrough that was achieved through intense interest 
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and attention to the airborne ASW problem. An example of the 
Motivation Rating Form is included as Appendix A. 

In March 1971, the form was tested at Naval Air Station, Moffett 
Field. The squadron (VP-19) had 18 Jezebel operators in the AW 
division and five senior petty officers familiar with each of the 
operators who were available to rate the men. The raters were 
asked to indicate when they were basing their evaluations on 
observation or an expectation. This was to attempt to separate 
actually observed behavior from an estimation that the man would 
reasonably be expected to perform in the described manner in an 
appropriate situation, After completing the form, each rater was 
asked to comment on his general attitude about the form, its 
acceptability, completeness and any specific comments about the 
description of motivation that it provided. Information from these 
interviews is the basis of part of the Analysis of Data section. 
Following the interview each rater was asked to rank order each of 
the men according to their mission-oriented motivation. They were 
asked to minimize the influence of the form they had just completed 
and to the best of their ability eliminate the influence of on the 
job performance. The information that was collected was analyzed 
using several statistical techniques that are described in the 


following section. 


B, SAMPLE POPULATION 
The population that was sampled in this project was drawn from 


United States Navy Jezebel Operators. Jezebel is a system for the 
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airborne classification and localization of submarines using 
information transmitted from sonobuoys. Specific details about the 
system and the techniques of analysis are classified, however, they 
involve a combination of training, experience, memory, mechanical 
and electronic devices and onboard reference publications. 

The minimum requirements to become an Antisubmarine Warfare 
Operator (AW), the rating which includes Jezebel operators, are a 
minimum General Classification Test (GCT) and arithmetic test (ARI) 
combination of 110, successful completion of an aviation physical 
examination and the applicant must volunteer for AW training 
(approximately 7% of the applicants are selected for training). 

There are less than 2,100 AWs on active duty in the Navy and of these 
about half are designated Jezebel Operators .+ The training that 

leads to assignment to a flight crew in a squadron takes nearly nine 
months and is in the top 25% of Navy schools on the basis of the 

cost of training. After assignment to a squadron, training continues 
until designation as a member of an Alfa Crew which indicates achieve- 
ment of a high level of operational readiness and completion of a 
squadron training syllabus. Acquisition of basic skills, maintenance 
of skill, and operational flying are a full time task. Because of 


this AWs are becoming more and more an autonomous group, freed from 


lihis information was made available by the Bureau of Personnel 
rating detailer. 
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most other duties except watch standing and certain military 
obligations. They are not trained in a secondary technical 


Specialty. 


ro 





IV. PRESENTATION OF DATA 


In the instructions for the raters they were asked to circle 
the number that best fit the man's behavior in the area described 
by the item. They were further instructed to complete one item for 
all men, then to proceed to the next item and to complete it for all 
of the men. 

In preparing the data for analysis the score on the items was 
totaled to give an overall grade. In some of the analyses the 
total score was used and in other sections the score on a particular 
item or an average score was used. Averaging scores was possible 
Since each man was rated by five independent raters. 

The scores that were recorded are included in the data 


section. 


A. INTERRATER RELIABILITY (IRR) 
The first analysis was a test for the interrater reliability 


of the form using the Pearson Product-Moment Correlation: 


_ _N=XY - (2X) (ZY) 
Vv [NEX: = (Ox) TNay = (rYI | 
where: 
Baise tne correlation 
X is the value of one variable 
Y is the value of the other variable 


N is the number of cases 
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The results of this analysis are listed in Table I. The data from 
which the correlations were computed are included as Appendix C. 
The statistical significance was computed for each correlation 


isinemence ‘Stuacnt'’s t'': 


(n-2) 


t = fr 
(1-r*) 


where: 

Paceule COmrebation. 

n is the number of cases. 

The second statistical test was a comparison of the interrater 
reliability based on the rank ordering that the raters were asked 
to do after completing the form and the rank ordering generated from 
the scores taken from the rating forms. For this comparison the 
Spearman Rank Order Correlation rho (9) was computed. Both sets of 
correlations are included in Table II for ease of comparison. The 


formula for computing the Spearman rho is: 


6=D- 


el = ea 
N(N” - 1) 


where: 
D is the difference in rank ordering for the two criteria 
being compared. 


N is the number of elements that have been ordered. 
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TABLE I 


interrater Reliability 


Correlation* b 

(between raters) n 
x * 

a2 .639 13 
= * 

1,3 2545 13 

M4 <i) Ores 13 
= “ 

is .617 14 

r, 3° Noc4 ** 13 

To 4 = ,586 13 
= k* 

Ns .931 14 
= * 

Tx 4 7 O44 13 

Tr. 5 = Boag 14 
= * 

aS e435 14 

Note.-- 


“The SUDSCrIpt indicates the specific rater 
involved, i.e., YT) 9 is the correlation between the 
3 


ratings of rater 1 and rater 2. 


De = 14 whenever rater 5 is involved because he 


was the division CPO, familiar with each of the Jezebel 
Operators but not qualified himself and, as such, not 
included in the sample. In all other cases the two 
raters being compared were excluded from the sample. 


ZoupmeercCant ae the .0S level” 
Avoigniereant ae the .O01 lever. 
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Interrater Comparison Table 


IRR 
(Rank Ordering) 


P12 = ,639** 
oe = .095 
P14 = .100 
P15 = .266 
2.3 = .56* 
2 ara 
Epes = 604= 
P34 = .123 
P35 = ,600* 
4.5 = .139 


foeenaticant at the 
faotoniiicant at the 


TABLE 1 I 


IRR 
(Form) 


SO 


20 5 


.O1 


. HAIG * 


,145°" 


95 


(SoZ 


Hy ae 


goozes 


sO LS 


.OS level. 
.O1 level. 


i 


aS 


1S 


13 


14 


13 


13 


14 


13 


14 


14 





As can be seen from Column 1 of Table II, there was only 
occasional agreement in rank ordering, however quite good 
agreement was achieved using the form. There was much better 
agreement among the raters when the form was used than when a 
global rank ordering was made. The reasons that the one rater, 
rater number four, tended to differ from the others can only be 
generalized from the situation and from other studies. (It is 
interesting to note that he was rated highly in motivation by his 
peers and he expressed very positive feelings about the form.) 
Perhaps the most likely explanation stems from the type of training 
that is done by the squadrons. After an operator completes the 
formal Navy school and is assigned to a Squadron the remainder of 
his training is conducted by the senior operators in the squadron. 
When a man is assigned to a crew his sphere of contacts is reduced 
Significantly. The result of all this (coupled with social inter- 
action) is an unequal familiarity among the operators. Many studies 
have been done on the reliability of human judges. The limitations 
and biasses are well known and include such phenomenon as "halo 
error,” "leniency error," "similarity error," and others (see, for 
example, Kelly, 1967). The situation that exists in a typical 
Squadron could easily amplify these errors and could distort or 
invert the relative rating given to some of the men by one of the 
raters. This is considered to be the most probable explanation 


for the one differing rater. 
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B. ITEM ANALYSIS 
The analysis of the responses to the individual items was 
undertaken using four general methods. First, the means and 
Standard deviations for all responses to each item by each rater 
were computed, this was then broken down into two groups; one by 
the military rank of the person being rated and another according 
to the rater. Then a correlation of the average score on an item 
with the man's total score was calculated. Table III contains the 
composite score, the breakdown by seniority and the correlation with 
total score. Table IV is a listing of the means for each rater. 
Table III can be used to determine the items that contribute 
most to the total score and discriminate well among the men. If 
an item describes a degree of behavior that is observed or expected 
in all men then it should have a high mean and a low variance and 
will not have the ability to discriminate between men. Therefore, a 
mean near the midpoint is desirable as is a variance at least large 
enough to indicate a spread of scores. Another desirable character- 
istic for an item is a high correlation with total score. During the 
interviews conducted immediately after the forms were completed it 
became obvious that the importance of one dimension had been over- 
looked; that was the differences in behavior attributable to 
different levels of experience. As an example of this difference, 
a motivated man undergoing training as an AW might show his 
motivation by spending extra time in the flight simulators or 


trainers, whereas an operator who had achieved a high level of 


ail 





Mean 
Item All 
3.65 
4.19 
3.69 


Beat) 
aS, 
.o6 
.09 
.63 
29 
83 
. 60 
aco 
se, 
045 
nus 


y9 OO 42 SS FF Roa Sa eG a HS oo 
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Note. cae 


“Correlation of the item average by all raters with total 


Var. 


All 


74 
.10 
72 
ay, 
.49 
i 
al 
pil 
. 80 
foU 
7 
ANS 


W BP MO W NYO fF DY NO NYO KH NO 


i990 
2.40 
Das 


scores by all raters. 


Mean 
Senior 


SB WN SB Ff HH WNW FW FP Wa fan ff fF Ff 


TABLE III 


.90 
207 
03 


OS 


cod 
1010, 
yall 
107 
200 
ly) 
74 
ae 
43 
.00 
Beil 
207 
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Item Comparison Table 


Var. 


Senior 


HH eB CO FHF WH DH H&B DY KH NY KF KY CO KH CO FF 


53 
92 
24 
73 
. 80 
.10 
38 
07 
eal 
.79 
.50 
37 


86 


94 
120 
94 


Mean 


Junior 


W MO WA WN DO FF NO KF WD DO WW WD WD We WwW 


sol 
41 
70 
49 
. 66 
9 
84 
Ao 
10 
woo 
» 88 
.49 
wo 
46 
94 
5S, 


Var. 
Junior 
evi) 
709 
Ze OS 
2.41 
2.50 
ad, 
1.45 
ZuoL 
B42 
1,64 
4,24 
2.94 
2305 
Zao 0 
Se 22 
1.96 


Corre- 
lation 
All 
Sood 
972 
BYE! 
942 
a2 
1076 
nou 
OTe, 
ys 26 
.880 
. 869 
621 
921 
884 
.780 
.955 





TABLE IV 


Rater Comparison Table 


Difference 
eee Between 

RRR Means 

Item 1 2 3 4 5 (H-L) 

A Mean 3.40 AS 3.73 S207 3.00 wa2s 
Var 2.85 1.80 0.64 1.05 1.87 

B Mean Acs Awe Silo as 4.00 ea 
Var. 252 176 1.35 Zed Varo | 

C Mean 4.80 3.81 S47 3.19 S.25 iow | 
Var. 1a 1.63 0.84 2.16 1&7 

D Mean 5Zz0 4.69 AT 3.56 3.69 1.60 
Way. 2.46 1.69 0.78 2220 2.25 

E Mean 4.93 4.37 5.20 2044 4.56 2576 
Var. 1.30 2e2 131 1.99 1.20 

F Mean 3.67 4563 5.00 3.69 4.50 1333 
Var. Dae 2eo7 1214 2.10 55 

G Mean 2.80 Des 2307 2.69 2.13 O275 
Var. 4.31 AS 2 Deel 4.50 5.98 

H Mean ow) 4.19 3.20 3.94 4.44 1.47 
Var. 2.81 2.96 1.03 1.80 1.20 

I Mean 4.60 2.94 2443 iS 1.44 Sik6 
War. 1.97 3.53 1.29 eS 4.13 

A] Mean 2287 4.00 3.00 3.06 3.51 1.13 
Vane 2uZ pay P24 S220 220 3.01 

K Mean blk 2.19 2.67 22556 1.63 3.50 
Var. 1.55 1.63 ---- 2.86 3.98 

5 Mean 3.80 3.69 DAT 3.06 0.06 3.74 
Var. 2.89 1.56 e27 0.73 0.06 


egg ng ag i pA A 


(Table continued on next page) 
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TABLE IV (continued) 


Difference 
Between 

Rater Means 

Item 1 2 3 4 5 (H-L) 

M Mean 3.60 4.06 Coal Seo oao4 0.46 
var, rae S555 1324 3.00 1.93 

N Mean 4.67 4.13 S375 3.81 450 phe by, 
Var. 67 2 2 0.50 2.10 1.07 

0 Mean 4.8 S206 SiS 3.69 2225 2255 
Var. 1.46 1.26 0.92 2.76 1.40 

P Mean 4.46 4.63 4.42 3.56 5054 157 
Var. 1.98 1.58 0.81 2.40 2605 
Avg. Mean UNL 7s S200 Ke 4 3.30 3.06 





competence would be unlikely to request any more trainer periods than 
the minimum monthly requirement. As a correlate, a motivated and 
experienced man might demonstrate his motivation by working to qualify 
in other crew positions, an activity that would not be expected of a 
junior man working toward his initial qualification. This differ- 
entiation by experience level existed in some areas, however it was 
not a characteristic of all items, many of the items were equally 


applicable to junior and experienced men. 
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The method used to evaluate items for their applicability to 
a particular experience group was to use the difference in the means 
of the two groups to identify items with a likelihood of having this 
characteristic. An item with a relatively high difference should 
favor the experienced group and a low difference should indicate an 
item of greater applicability to the lower experience level. (In no 
case was the mean of the senior operators lower than the mean of the 
junior operators. The average difference in the mean was 1.4 so that 
smaller differences tend to identify items more characteristic of 
junior men.) | 

Another characteristic of a desirable item is that it should be 
marked in a Similar range by all raters when rating the same group. 
Table IV shows the mean score on each item by each rater and the 
difference between the high and low score. A large difference tends to 
jeentify items that have been interpreted differently by different 
raters. An exception to this logic occurs when one rater has a mean 
which is much different from the others since either a high or low mean 
acts to limit the possible scores in one tail while increasing the 
possible scores in the other tail. In this case a slight change in 
the wording may greatly improve the suitability of the item. One 
rater, the rater whose interrater reliability was low, tended to differ 
markedly from the others at least twice as often as did any other rater. 

To illustrate how some of these desirable characteristics have been 


applied to the items, compare the figures in the last column of Table IV. 
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Three of the items have differences between means greater than 3.00 
(items I, K, and L). On examination of these items it seems logical 
to assume that the interpretation could be different for the different 
raters. Probably item I is the item most susceptible to "extension 
into the future.'' That is, evaluating the phrase in the instructions, 
"if the circumstances were appropriate,'' to include situations 
considerably beyond that which was expected or interpreted by most 
Pawers . 

The wording of item K is such that it does not adequately describe 
or differentiate a highly motivated man. As it is written it could be 
broadly interpreted to include the majority of sailors and this 
apparently was the interpretation made by some of the raters. Item L 
was very much subject to the experience element described earlier. 
Limiting this item to evaluation for a particular experience group 
should ease the problem of interpretation. 

From this analysis a new rating form was developed and is included 
as Appendix B. This form should have greater interrater reliability, 
better discrimination between highly motivated junior men and the 
senior men whose motivation is marginal and it should be even more 
acceptable to the users. In preparing this form, items with a narrow 
variance or a very high or low mean were either modified or eliminated 
and the information from Table IV, together with a "face" determination 
was used to separate items by their applicability to a particular 


experience group. 
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CC. POLICY CAPTURING 

Certain exploratory attempts were made to understand the criteria 
the raters used in the free rank-ordering. These efforts at policy 
capturing were rudimentary and not originally planned for inclusion 
in this study. The working concept was that in general raters tend 
to make evaluations based on a limited number of elements even if much 
more data is available (Whisler §& Harper, 1962). Because of this it 
seemed possible that the rank-ordering was made on the basis of only 
a few types of behavior and, if this was the case, they should be 
among the incidents described in the items on the form. To test for 
any primary relationship a scatter plot was drawn for each item showing 
the average score that the man was given by the five raters on the iten, 
plotted against his average rank order. This was largely unsuccessful, 
although two items (O and B) did have a plot somewhat similar to the 
expected shape. If a strong correlation had been found, it might have 
pointed to a simplification of the rating technique or of the definition 
of motivation. 

This analysis was undertaken to determine if further, more exhaustive 
techniques such as factor analysis were warranted. Because of the weak 
correlation and the limited data no further investigation was undertaken 


at this time. 


D. PROBLEMS AND UNANSWERED QUESTIONS 
In the conduct of the project several problems came up which deserve 
discussion. First was the question of the character of motivation in 


the population sampled. Does the full spectrum exist from highly 
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discussion. First was the question of the character of motivation in 


the population sampled. Does the full spectrum exist from highly 
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favorable motivation to strongly negative motivation, or is the range 
limited (because of the selection procedures) to the area from neutral 
to highly favorable motivation? During the interviews it was impossible 
to generate neutral or negative incidents. It is difficult to 
conceptualize examples of low or negative motivation other than 

apathy, disinterest, or, in the extreme, destructiveness. This may 

De, im part, related to the questioning or interviewing technique 

used where positive or favorable situations were set up initially. 

If the procedure were reversed it is possible that neutral or negative 
incidents could have been develoved, 

One problem that is frequently perplexing in the analysis of 
evaluations 1s the so-called “halo effect," the propensity for raters 
who know a man who should be rated high in one area (for example, 
promptness) to rate him high in other areas for which they do not 
have an adequate basis for evaluation. It can be expected that this 
would occur on a motivation evaluation. However, what may appear to 
be "halo error" is expected if each item is really measuring the same 
thing. Secondly, such an error may not be detrimental to the ultimate 
result in the early stages of development if certain necessary behavioral 
descriptions have not been included. This possibility should be 
alleviated when and if motivation is broken down to dimensions which 
can be put on scales. While motivation is very likely multidimensional 
the detailed investigation of these dimensions might best proceed from 


amernattally global definition or treatment. 
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A clear and present problem in the measurement of any complex 
characteristic is the validity of the method chosen. This problem 
is a continuing one and one which can only be decreased with extensive 
and lengthy techniques. Further, it is intensified in the early stages 
of test development. There are at least two types of validity which 
the Critical Incident Techniques insure. First is what might be 
termed source validity; it is a valid measure of motivation because 
it was developed from descriptions of motivated behavior. To a 
degree this same fact can be considered to imply consenual validity 
and the consensus was peeien guaranteed by the post-test interviews. 
Face validity is at best an insufficient test, however, the form as 
administered certainly does have face validity. 

To improve the validity, to generate a numerical value for the 
validity of this instrument for measuring motivation will pose certain 
problems. First and most imposing is the problem that arises from the 
fact that this investigation was at the limit of current understanding 
of motivation. Because of this there are no other equal or similar 
intermediate criterion against which this can be compared. However, 
certain schemes can be envisioned which might involve setting some 
Situation or series of situations in which a motivated man might be 
expected to perform in a particular fashion. These concepts are only 
fragmentary and conjectural and would require a great deal of additional 


thought and testing before they could be attempted. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


The most significant conclusion that can be drawn from this study 
is that an instrument was developed that improved the reliability of 
ratings of a person's motivation. Since this work was in an area that 
has been virtually untouched, it was impossible to relate the results 
to any other findings for comparison. 

A second conclusion that seems to be warranted is that such an 
instrument would be applicable across a wide range of different skills 
and tasks with only minor modification. This conclusion was supported 
by the very general nature of the items that were generated in the 
interviews even though special attention was given to elicit and 
report incidents with the highest possible degree of specificity. 

This characteristic, while not completely unexpected, was encountered 
to a Surprising extent. It is considered that a very few different 
forms could be developed to cover all but the most extraordinary of 
occupations or individuals. This conclusion will require additional 
Study to verify. 

It is certainly not expected that a strong relationship exists 
between task or job motivation and career motivation. However, it does 
seem likely that a low task motivation should help to identify people 
who would be less desirable to retain in the military. 

There is a need to look at the variation of motivation over time 
and to test this variation with performance. The time dimension is 


one which has not been investigated. 
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The low Pe onenimnnerween morale or job satisfaction and 
production is well known, however, it seems possible that there may 
be a more complex interrelation with motivation being a key element. 
This possibility can be investigated through a motivation form. 

One of the most intriguing possibilities is the use of these 
findings and the methods that were used to discover and to 
appropriately value incentives. The relationship between the 
behavior that was reported as demonstrative of motivation and 
incentives that might be useful in motivating workers may at first 
Scen remote, Ave aay times during the interviews the situation 
was an important element in the display of motivated behavior. It 
was often the case that the situation seemed to be the motivating 
factor and as such should be viewed as an important incentive. 

These descriptions were strongly supportive of Herzberg's ''Two 

Factor Theory," the most profound motivation came as a direct result 
of doing the job the men are trained to do. During the interviews 

it became obvious that problems in communicating ideas and value 
systems from the lowest levels in an organization to levels where 
meaningful action can be taken are still very much the rule. Attempts 
at uncovering valued incentives should help to bridge this gap. 

Most personnel evaluation systems require a rating of personal 
attributes or personality variables and the value of these variables 
to an organization is highly subjective at best. Personality traits 
are extensively used in evaluating the performance of people who are 


far removed from a direct influence on production. It may well be that 
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motivation is a trait that is meaningful to an organization and can 
be measured more precisely, reliably and more objectively than many 
of the traits currently being rated. There is the added possibility 
that it will have greater correlation with relevant performance than 
the other traits. 

The development of a well tested, reliable and accurate measurement 
technique for motivation and one that is relatively free from the 
effects of performance would provide a powerful new tool for further 
investigation of numerous concepts of management policy and leadership 
techniques. It is possible that changes in motivation might be a much 
more valid measure of leadership and management ability than output or 
performance in fields where ultimate criteria can only be inferred or 
that are strongly influenced by equipment differences or situational 
aspects which are not easily assessed. 

Certainly this project has done one thing and that is to generate 


an objective operational definition of task-oriented motivation. 
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APPENDIX A 


MOTIVATION 


Background Information: This evaluation form is an attempt to develop 
an objective measure of motivation. The items that are included have 
been taken from descriptions of highly motivated men. 


Directions: Circle the number that best indicates how characteristic 
the item is of the man being rated. The evaluation should be based 
upon direct observation wherever possible. When direct observation 
has not occurred, indicate whether you think the man would act in the 
manner described if the circumstances were appropriate. This evalu- 
ation is directed at mission or job-oriented motivation only. 


Item Frequently Occasionally Never 
os 4 Ss 0 





A. Asks probing important questions; 
questions that are difficult to ans- 
wer and often require special effort 
Or research. ce Soames 5 2. 71 0 


B. Doesn't wait to be told to do 
something, sees things that need 
to be done and does them. 6 S 4 3 Z 1 0 


C. Seeks out other operators that 

you consider to be motivated, enjoys 

being around professionally competent 

men. Avoids poorly motivated men. 6 v5) 4 ce 32: 0 


D. Has taken time out to help some- 
one with a problem. 6 5 4 Small 0 


E. Discusses advancement and profi- 
ciency exams with other operators at 
length shortly after taking them. 6 5 4 Soe 1 0 


F, Watches and assists mechanics 

GG cecinicians in the repair of 

equipment to help them and to 

improve his own knowledge. 6 5 4 Sue. alee 0 
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Item 


G. Has given lectures ans has pre- 
pared lesson plans when he was not 
required to do so. 


H. Has gone to extra effort to get 
information or to prepare himself 
ahead of time to help make a flight 
go smoothly. 


I. Has originated improvements in 
equipment or procedures. 


Jue Has shown mmterest inethe other 
crew positions in the aircraft 
and has taken positive steps to 
qualify in one or more. of them. 


K, Has come back to the 
squadron from TAD or even from 
leave to attend an important 
Heeture or to complete an 
important task. 


L. Spends more than the 
minimum time in the trainers. 
Asks for additional trainer 
periods. 


M. Has contacted other 
squadrons, the Wing, the ASCAC 
or FAETUPAC to get any new or 
relevant information. 


N. Was reluctant or hesitant to 

leave an important or interesting 
Saemation directly related to the 
ASW mission. 


O, Starts the preflight checks of 
equipment prior to the normal time 
in order that any problems encoun- 
tered Can be corrected. 


P, Has stayed on a job until it 


was finished regardless of regular 
working hours. 
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Frequently Occasionally Never 


6 5 4 5 2 1 0 
6 5 4 5 Z i 0 
od Ss <2 1 0 
00) et Se 1 0 
6 5s 4 3 2 ] 0 
6 5 4 3 Z 1 0 
Of 55). 4 3 2 1 0 
6 5 4 i 2 1 0 
SH bo 4 See i 0 
6 5 4 5 2 1 0 
Cio 4 5 2 i 0 





The following items can be answered by either Yes or No. 


Item 





Q. Has learned a special procedure on his own 
from manuals and publications and is considered 
to be an expert in this area by the personnel in 
the squadron and, perhaps in other squadrons as 
well. 


R. Is interested in making sure that important 
intelligence information is sent to higher 
levels in the information chain. 


S. Has an eagerness to find and share new information 
with other squadron personnel. 


T. Assumes that the information that is being 
developed on a mission is important and will be used 
later on by someone who wasn't there when it was 
collected; keeps his logs and grams accordingly. 


Name of man being rated 


Yes 


Yes 


Yes 


Yes 


No 


No 


No 


No 





Rater's initials 
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APPENDIX B 


MOTIVATION 
(REVISED) 


Background Information: This evaluation form is an attempt to develop 
an objective measure of motivation. The items that are included have 
been taken from descriptions of highly motivated men. 


Directions: Circle the number that best indicates how characteristic 
the item is of the man being rated. The evaluation should be based 
upon direct observation wherever possible. When direct observation 
has not occurred, indicate whether you think the man would act in the 
manner described if the circumstances were appropriate. 


The first five items (A-E) should be completed for junior men only. 
Junior in this case should be considered as not having achieved Alfa- 
Crew status. 


Item Increasingly Favorable 
0 il 2 3 4 5 6 


A. Asks probing, important questions; 

questions that are difficult to answer 

and often require special effort or 

res earch . Qn -—-b 2553 aoe 


B. Spends more than the minimum time 
in the trainers. Asks for additional 
Preanenr Periods , OO? b 2°25 stGiep oe 30 


C. Seeks out other operators that 

you consider to be motivated, enjoys 

being around professionally competent 

men. O 1 2 3 4 5 6 


D. Starts the preflight checks of 
equipment prior to the normal time 
in order that any problems encoun- 
Cepedeean be corrected. 0O 1 2 3 4 5 6 


Es Has gone to extra effort to get 

information or to prepare himself 

ahead of time to help make a flight 

go smoothly. 0 1 2 3 4 5 66 
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The second five items (F-J) should be completed for senior operators. 


Increasingly Favorable 
Item 0 il 2 3 4 S 6 


F. Has given lectures and has prepared 
lesson plans when he was not required to 
do so. OO Ll A234 45s 


G. Has come back to the squadron from 
TAD or even from leave to attend an 
important task. OG. 1 VEZ RES eed: -- “Seo 


H. Has originated improvements in 
equipment or procedures. 0 1 @53° 4 @& 


I. Has shown interest in the other 

crew positions in the aircraft and 

has taken positive steps to qualify 

in one or more of them. OF 1. i252 A aso 


J. Has taken time out to help someone 
with a problem. 0» Te) 2 + (eee 5S eG 


The remaining items should be completed for all operators. 


K. Has contacted other squadrons, 
the Wing, the ASCAC or FAETUPAC to 
get any new or relevant information. Om 2 5a eee ee 


L. Was reluctant or hesitant to 

leave an important or interesting 

Situation directly related to the 

ASW mission. 0 il 2 5 @ 5 6 


M. Discusses advancement and profi- 
ciency exams with other operators at 
Wemeenesnontly after taking them. QO IL 2°93 4 255556 


N. Has stayed on a job until it was 
finished regardless of regular working 
hours. oF 1 2.3 4. “Seo 


O. Watches and assists mechanics or 

technicians in the repair of equip- 

ment to help them and to improve 

his own knowledge. He 1 2 3 4 oS 
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nein Increasingly Favorable , 
0 i 2 3 4 5 6 


P, Doesn't wait to be told to do 
something, sees things that need to 
be done and does them. 0 1 2 3 4 5 6 


Q. Is interested in making sure that 

important intelligence information is 

sent to higher levels in the information 

chain. 0 (1) 2.50 oY 5. 6 


R, Has an eagerness to find and share 
new information with other squadron 
personnel. OO de 2-354 ese G 


S. Assumes that the information that 

is being developed on a mission is 

important and will be used later on 

by someone who wasn't there when it 

was collected; keeps his logs and 

grams accordingly. 0 1 2. 3. Aes oe 


Name of man being rated 


Rater's initials 


58 





APPENDIX C 


DATA SECTION 1 
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DATA SECTION 1 (continued) 


M 


m mM wo mM ce I N 
t mA M4 So) t ™N 
t m wm N © | re 


40 


W 


(continued on next page) 





DATA SECTION 1 (continued) 
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DATA SECTION 1 (continued) 
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DATA SECTION 3 


aS... ee rg re a re le fl SS 
a a ae a a a TS ad 





eee Average Score on Item 

Score A B C D E F G H 
BS 69 .00 4 5a) S.2U0 4.40 4.50 hes 7 S Se. 7S 4.20 
B., 39.75 fa is 5250 3200 2.40 3.50 Aree 50 S75 
C, 22.80 2.20 2.00 2.40 1.80 3.20 1.60 0.00 1.40 
C, 69.40 4.00 5.40 4.80 SiO 5.40 5 #40 4520 4.40 
C. 54.00 4,40 4.00 4.00 4.40 SOU 4.00 . 80 3.80 
FE S440 3.40 4.40 3.60 4.20 4.60 4.40 2.60 4.00 
H 68.00 3.60 4.80 4.40 5.00 5.40 5.20 4520 420 
Ky 50.60 3.40 4.20 3.60 3.40 4.60 4.40 1.60 4.20 
K, 55.40 4.80 4.00 4.60 4.60 4.50 2-60 2.60 4.60 
Ke 54.20 5.20 5.00) 3.40 4.40 3.80 4.20 2.00 4.40 
Ey 767240 4.60 S260) 3.60 5.60 S240 5.60 5 a0 5.40 
M 50.00 3.40 4,00 3.40 4.00 4.00 3.80 1240 3.80 
Pe 29340 2.40 2.00 2320 5-00 220 2.40 . 60 2.00 
R 45.80 3.00 3.80 5.40 3.60 3.60 4.00 40 3.80 
Tae 4 00 5 20 5.60 4.20 5.60 4.20 5.60 5 40 2270 
W 67,20 4.00 5.00 3.80 5.00 5.60 5.40 5260 4.60 


ae Average Score on Item 

Score if J K L M N 0 P 
BS 69.00 3.00 4.40 3.00 2275 4.50 4.50 3.50 525 
B., 39.75 0.75 2.00 1.00 2.00 S500 5275 Sc 2267 
C, 22.80 0.00 1.00 On75 ees: 1.00 2.00 TSU 225 
C, 69.40 Ries 5515 4.80 2.80 4.80 5.00 4.50 4.80 
C. 54.00 Ih egas3 ois CR ES 3.00 3.80 4.20 3.60 4.40 
Pe 54-40 2.80 4.00 20) 1.60 3.40 3.60 2275 3.60 
H 68.00 3.80 oe 3.60 2.20 4,00 4.40 a220 See 
Ky 50.60 2525 3.00 2.00 2.60 3.00 3.80 3.00 3.40 
K, 55.40 1375 Pa S15 4.20 4.50 52s 4.60 4.20 
Ke 54208, 72. 20 LINAS 2.50 2.20 3.75 4.00 Sac2 0 4.20 
L 76.40 4.40 5.20 4,00 S200 5.60 4.60 AT2S 5.60 
M 50.00 22295 A525 2225 2200 4.00 3.80 3.00 S31OU 
P 29.40 LSC 2.00 1.50 1.60 1.50 Zee 1.50 2225 
R 45.80 2.00 2725 2aoU 2.40 5.7 5 5275 StS 3.40 
li 74.00 4.20 5a 5.20 5.20 5. 20 5.40 3.80 5220 
We 67.20 3.80 4.00 Sa he S420 4.50 aS Ls, 4.20 


a ec rrp es ae a SSS SS PY SP 
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